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e [.]: non-linear, not QP anymore :-( 
—what about dual? kernel? 


e cannot distinguish small error (slightly away from fat boundary) 
or large error (a...w...a...y... from fat boundary) 
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e record ‘margin violation’ by £,—linear constraints 


e penalize with margin violation instead of error count 
—quadratic objective 
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e record ‘margin violation’ by £n 
e penalize with margin violation 
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e parameter C: trade-off of large margin & margin violation 
* large C: want less margin violation 
e small C: want large margin 


e QP of d+ 1 + N variables, 2N constraints 





Dual Problem 
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—only difference to hard-margin: upper bound on a, 
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Messages behind Soft-Margin SVM 
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Soft-Margin M Algorithm 
© qnm = ynymK(Xn, Xm); p = —1w; (A, c) for 
equ./lower-bound/upper-bound constraints 
© a + QP(Qp, p, A, c) 
© b+? 
© return SVs and their a, as well as b such that for new x, 
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SV indices n 





Egia Br sibyg? fEHard-Margin SVM Dual}, complementary 
slackness&f't: On (1 — y. D! zn + b)) = 0, 1€8JSV, Bla, > OW, 34S 
Fb = ys — W" zs 


ABZ, ftSoft-Margin SVM Dual, TB gcomplementary slackness FT (A 
HANARA Fan Men) : 


o (1 m lm ya (w? zn + b)) =0 
Pata Ges (C — On, )E = 0 


HSV. Bla, > OBR, FEN, ATE, ERECTA DAE. TRESS 
complementary slackness&(4, WRSC — an £0, Bla, AC, WJ—xEf3 

En = 0, (&AXISS—" complementary slacknessz&ft, Blais) 

b = ys al zs. WEB < o, < CHIARI free SV, SIAR RUS, Lë 
73: 


b = Y; — ` Om Un K (25 , ta) 
SV 


ARKEEN MRR EA, < C, Gre ERER? ADIRE free 

SV, Ba: AF SHR Ea, = CEAD? iat, ELFE—ASViE 

a, < CAYMZE RAN. UMREM A free SVEN, BEA DbISS EUR E 
SEI, (MATH, Lol Pi EKKTR AAS ANAA 
LAT. Gët HE, TA, 


hard-margin SVM soft-margin SVM 


complementary slackness: complementary slackness: 
an(1 — Yn(w"z, + b)) =0 an(1 — En — Yn(w"Z, + b)) =0 
(C — an)£n =0 
e SV (a; > 0) * SV (os > 0) 


=> b = ys — Keis — W 2s 
* free (os < C) 
= Ce zU 


>b=y,—w'z, 





RK, REACHER margini. (IAE, xXj--Soft-Margin Gaussian 
SVM, C435IBX1, 10, 1008+, *8nzAYmargingd KAA: 





NEB, C-1Hj, marginE 4H, (Drëpse, HCHK 
BRE, marginthekeH, ED DECK, IESHBUIBIT HJ, CERAT 
margins 2S1ERSRS—^ Ma. Civ JN, GERIT] T (SUB margin, Gelee 
RAJA, CIRCA, PURPA SRSA IE, -rnJmarginf&Z, FARM, 4CH 
(RARE, BADR, (ARR EnoisetBiAiT f 4HHB, Hogg 
WE. teetitSoft-Margin Gaussian SVMF RES HHA, PASZ 
(y, CU Ice, 


IIe ke Son, WARS MBS. EHO <a, < CENTS 


complementary slackness2&ft : 
On (1 — En — Yn(w? zn +b)) = 0 
Bb = (C — an) = 0 
Aa, — 0, Bén — 0. En USECH AED, an = ORMAZRRESV. ATLA 
ATMA RAEmarginZ At (të tEmargine) , EIT9ZT2S1ETRR. 


#0 <an « C, Bén —0, Bn, (wl zn +6) = 1. & = ORNARE, 
Yn(w" zn +b) = lmiAeaEmargin. E, ixi&gxBpfree SV, HAT bI) ei. 


fa, =C, FARES, BEE, BSS — y, (wI zn +b) = En, IRET 
Armnrsmarginivize, EnA, mamargi ERA. RAE, = OW, rR 
wWiEmargink, Pir IM IMA RAtEmarginz AARD (aketemargink) , 6 
PREAH A DIR. XH AeKAbounded SV, 


PALA, fESoft-Margin SVM Dual}, Ria, WRU, sto LASER ESTE SS TRIB f 
hd, 


es non SV (0 = a4): En = 0, 
‘away from’/on fat boundary 
es O free SV (0 < an < C): £5 = 0, 
on fat boundary, locates b 
e A bounded SV (a, = C): 
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